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ABSTRACT 


This  note  presents  the  formulation  of  a  class  of  optimization 
problems  dealing  with  selecting,  at  each  instant  of  time,  one  measure¬ 
ment  provided  by  one  out  of  many  sensors.  Each  measurement  has  an 
associated  measurement  cost.  The  basic  problem  is  then  to  select  an 
optimal  measurement  policy,  during  a  specified  observation  time  in¬ 
terval,  so  that  a  wei  ghted  combination  of  '’prediction  accuracy”  and 
accumulated  ’’observation  cost”  is  minimized.  The  current  analysis 
is  limited  to  the  class  of  linear  stochastic  dynamic  systems  and 
measurement  subsystems.  The  problem  of  selecting  the  optimal  measure¬ 
ment  strategy  can  be  transformed  into  a  deterministic  optimal  control 
problem.  An  iterative  digital  computer  algorithm  is  suggested  for 
obtaining  numerical  results.  It  is  shown  that  the  optimal  measurement 
policy  and  the  associated  ’’matched”  Kalman-type  filter  can  be  precomputed, 
i.  e.  specified  before  the  measurements  actually  occur. 
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1. 


INTRODUCTION 


There  are  many  engineering  situations  in  which  a  variety  of 
possible  measurements  can  be  carried  out  on  a  physical  system  or 
process.  However,  there  are  also  physical  constraints  that  impose  the 
requirement  that,  at  each  instant  of  time,  one  is  able  to  use  only  one  out 
of  a  possible  total  of  M  sensors.  In  such  cases,  one  has  to  make  a  decision: 
Which  measurement  to  make  at  present,  and  when  to  make  an  alternate 
measurement.  It  may  also  happen  that  one  can  associate  with  each  type 
of  measurement  a  per  unit-of-time  ’’measurement  cost”  reflecting  the 
fact  that  some  measurements  are  more  costly  or  difficult  to  make  than 
others,  although  they  may  contain  more  useful  or  reliable  information. 

This  type  of  problem  arises  in  the  following  types  of  systems: 
Telemetry- Data  Aerospace  Systems.  Consider  a  space  vehicle  and 
suppose  that  it  contains  M  sensors  each  of  which  measure  a  different 
signal  (i.  e.  yaw,  roll,  pitch,  or  perhaps  their  rates).  Suppose  that  one 
has  a  single  bandlimited  telemetry  link  so  that  at  each  instant  of  time  one 
can  communicate  to  the  ground  the  data  signal  from  only  one  of  the  sensors. 
Furthermore,  suppose  that  the  ground- control  can  command  which  sensor 
output  is  to  be  communicated.  How  should  the  ground  command  the 
telemetry  system  to  sequence  the  data?  What  are  the  criteria  that  the 
ground- control  can  use  in  order  to  make  such  decisions?  We  should  like 
to  stress  that  this  is  not  the  multiplexing  problem.  Each  sensor  can 
provide  a  group  of  measurements  which  can  be  multiplexed  for  transmission 
to  the  ground.  However,  we  assume  that  not  all  measurements  can  be 
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transmitted  to  the  ground  at  once*  Hence,  we  can  transmit  a  group  of 
measurements  now,  some  other  group  later  on,  and  so  on. 

Tracking  and  Discrimination:  Consider  the  observation  of  a  target 
by  a  radar.  Suppose  that  the  radar  has  the  capability  to  transmit  over  a 
specified  interval  at  each  instant  of  time  one  out  of  M  possible  waveforms 
and  that  each  one  can  be  used  for  making  a  different  type  of  measurement 
(e.  go  tracking,  Doppler,  wake  measurements,  etc,  ).  There  may  be 
different  radar  power  requirements  for  each  signal;  furthermore,  there 
may  be  different  computational  overhead  and  real-time  requirements 
associated  with  the  data  processing  of  each  type  of  return.  If  the  defense 
is  interested  in  discrimination  and  impact- point  prediction,  how  does  one 
decide  which  waveform  is  to  be  transmitted  at  each  instant  of  time? 

Socioeconomic  Problems  :  In  many  socioeconomic  problems  one 
can  make  measurements  by  assigning  a  group  of  people  to  collect  data 
or  carry  out  polls  or  look  up  statistics  etc.  How  does  the  manager  of  an 
information- gathe ring  subdepartment  allocate  his  data- colle cting  resources 
at  each  instant  of  time  so  that  an  accurate  forecasting  of  future  trends  of 
the  entire  company  or  government  agency  can  be  carried  out? 

The  above  situations  (and  many  others)  represent  typical  situations 
in  which  the  allocation  of  limited  measurement  resources  is  important. 

They  can  be  abstracted  in  the  following  context: 

1)  A  stochastic  dynamic  system  (satellite,  target  vehicle, 
socioeconomic  system)  is  involved. 

2)  One  cannot  measure  all  of  the  significant  variables 
(state  variables)  of  this  system. 
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3) 


At  each  instant  of  time  one  has  the  choice  of  making 
only  one  out  of  many  possible  measurements 

4)  Each  type  of  measurement  may  have  an  associated 
measurement  cost  per  unit  time 

5)  Each  measurement  is  unreliable  (noisy). 

6)  Usually  a  prediction  (forecasting)  is  involved, 
whose  accuracy  will  depend  upon  the  judicious 
choice  of  a  measurement  policy. 

7)  There  exist  tradeoffs  between  total  cost  of 
measurements  vs  prediction  accuracy. 

The  purpose  of  this  note  is  to  formulate  this  class  of  problems  for  the 

simplest  possible  class  of  problems;  the  assumptions  we  make  are: 

a)  the  dynamics  of  the  process  are  linear. 

b)  each  signal  available  for  measurement  is  a 
linear  combination  of  the  process  state  variables. 

c)  each  measurement  is  corrupted  by  white  noise. 

Somewhat  related  problems  have  been  studied  before,  although 
the  availability  of  results  has  been  relatively  scarce.  Athans  and 
Schweppe  [l],  [Z]  h  ave  studied  the  problem  of  the  timing  of  a  measurement, 
under  constraints  onthe  available  measurement  energy.  Specific 
applications  were  given  by  Schweppe  [3]  and  Schweppe  and  Gray  [4]. 

Control  problems  with  costly  observations  and  pertaining  to  the  timing 
of  observations  have  also  been  studied  by  Kushner  [7],  Vandelinde  and 
Lavi  [8]  and  Sano  and  Terao  [lO],  The  optimal  control  of  systems  with 
observation  constraints  has  been  studied  by  Meier,  Peschon,  and  Dressier 
[9].  The  studies  reported  in  the  literature  differ  from  the  problem  under 
consideration  in  this  report  due  to  the  fact  that  alternate  observation  policies 
dealing  with  allocation  of  costly  resources  have  not  been  explicitly  considered. 
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The  structure  of  this  note  is  as  follows.  In  Section  2  we  define 


precisely  the  plant  under  consideration,  the  assumed  statistics,  the  sensor 
constraints,  and  the  measurement  constraints.  An  optimization  problem  is 
defined  in  Section  2.  7  which  involves  the  selection  of  the  optimal  observation 
policy  and  the  associated  prediction  algorithm  so  that  a  weighted  combina¬ 
tion  of  prediction  error  and  measurement  cost  is  minimized.  In  Section  3 
it  is  shown  how  such  problems  can  be  attacked  using  the  Kalman  Bucy  [5] 
filtering  and  prediction  framework.  This  leads  to  an  alternate  formulation 
of  the  basic  optimization  problem  in  Section  4;  the  new  optimization  problem 
involves  the  minimization  of  a  deterministic  cost  functional  with  matrix 
differential  equation  constraints.  In  Section  5,  the  matrix  minimum  principle 
[6]  is  used  to  deduce  the  necessary  conditions  for  optimality.  Section  6 
suggests  a  computational  algorithm  to  solve  the  two  point  boundary  value 
problem.  Section  7  contains  a  discussion  of  the  off-line  and  on-line 
computational  requirements. 
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2.  PROBLEM  FORMULATION 

In  this  section  we  shall  summarize  the  basic  definitions,  notation, 
and  assumptions  which  relate  to  the  problem  under  consideration. 

2.  1  Plant  Dynamics 

Consider  a  linear,  possibly  time- varying,  plant  described  by 
the  stochastic  differential  equation 

X  (t)  =  A{t)x(t)  +  B(t)u(t)  +  £(t)  :  iS(tQ)  =  Xq  ^2  1) 

We  assume  that: 

.  The  plant  state  vector  x(t)  is  an  n- dimensional 

column  vector 

.  The  plant  control  u(t)  is  an  r- dimensional  column 

vector;  we  assume  that  u(t)  is  deterministic  and 

known  for  all  t  ^  t  . 

o 

.  A{t)  is  a  known  deterministic  nxn  time-varying 

matrix. 

B(t)  is  a  known  deterministic  nxr  time-varying 

matrix. 

^  d 
x(t)  = 

.  The  plant  driving  noise,  ?(t),  is  an  n- dimensional 

column  vector.  We  assume  that  ?(t)  is  a  white  noise 
process  with  zero  mean  for  all  t,  i.  e. 

E{£(t)]  =  0  for  all  t  (2.2) 

and  known  covariance  matrix: 

cov[£(t);£(T)]  =  E{£(t)£'(r)}  =  S  (t  )  6(t-r)  (2.3) 

The  nxn  matrix  S(t)  is  assumed  known  for  all  t^t  . 

—  o 

It  is  a  symmetric  positive  semidefinite  matrix  i,  e. 
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Fig.  1.  Block  diagram  of  the  linear  stochastic  plant. 


Fig.  2.  Visualization  of  the  multiple  sensors 
and  of  the  possible  measurements. 
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^(t )  =;^  '(t)  s  0 

The  initial  plant  state  x  is  not  known.  It  is 

— o 

modelled  as  a  vector- valued  random  variable 
with  known  mean  x  ,  i.  e. 


Efx  ]  =  X 
o  — o 

and  known  covariance  matrix  (symmetric, 
positive  semidefinite) 


covfx  ;x]=e{(x  -  x)(x  -  x)n=5] 

*-  Q  Q  ^  Q  Q  '  ^  Q  Q  Q 

E  =Z'  ^  0 
— o  —  o  — 

x^  is  independent  of  ?(t)  for  all  t  ^  t^,  i.  e. 
covFx  ;  5(t)]  =  0  for  all  t^  t  . 

Figure  1  illustrates  a  vector  block-diagram  of 
the  plant  under  consideration. 


(2.4) 


(2-5) 


(2.6) 

(2.7) 


2.2  Sensor  Constraints 

Let  us  suppose  that  we  have  available  M  sensors  which  can  carry  out, 
not  necessarily  independent,  measurements;  This  situation  is  illustrated  in 
Figure  2.  We  shall  let  ^j(^)  denote  the  measurement  vector  (set  of  signals) 
obtained  from  the  j-th  sensor  at  time  t. 

We  shall  assume  that  the  sensor  measurement  vector  z.(t)  is  an 
m. -  dime nsional  vector  given  by 


j=l.  2,  .  .  .  ,  M  (2.  9) 

J  J  J  J  J 


where  output  signal  defined  by 

yj(t  )  =  Cj  (t)x(t) 


(2.10) 
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Fig.  3.  Modelling  of  the  sensor  structure  and  sensor  noise. 
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and  C.(t)  is  a  known  m.xn  time-varying  matrix,  for  each  j, 

- J  J 

In  Equation  (2.9)  measurement  noise  associated  with 

the  j-th  sensor.  We  assume  that  each  ®j(^)  is  a  white  noise  process  with 


zero  mean 


Er^.(t)]  =  0;  j  =  l,  2,  .  .  .  ,  M;  t^t^ 
and  known  (symmetric,  positive-definite)  covariance  matrix 


(2.11) 


cov[e.(t);  =  E{e.(t)^'  (T)]  =  0.(t)6(t-T) 

J  J  J  J  J 


(2.12) 


0.(t)  =  0/(t)  >  0 
-J  -J 


(2.13) 


We  assume  that  each  noise  process  is  independent  of  ^  and  ^(t),  i.  e. 


cov[x^;  ®j(i)]  “  i  ^  j“i>  2,  ,  .  .  ,  M  (2.14) 

cov[?  (t);  9  (r)]  =  0,  for  all  t,  r,  all  j  =  l,  2 ,  .  .  .  ,  M  (2.15) 

The  measurement  noises  0-,  (t)  and  0.(t)  may  be  dependent. 

— K  — J 

Figure  3  indicates,  in  block  diagram  form, our  assumptions  regarding 
the  structure  of  each  sensor. 

2. 3  Discussion 

It  is  important  to  realize  that  our  assumptions  imply  that  each  sensor 
can  provide  a  group  of  noisy  measurements.  For  example,  suppose  that  we 
deal  with  the  attitude  of  a  satellite  whose  state  variables  are 
CO  =  roll  angle 
\lf  =  yaw  angle 
9  =  pitch  angle 
cb  =  roll- rate 
’If  =  yaw -rate 
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Fig. 


z,{t) 


ZgCt) 

o 


O 


18-3-13534 


SELECTED 

\ 

Q_  X 

DATA  SIGNAL 

MEASUREMENT 

SELECTOR 


4.  Visualization  of  the  selected  signal  z(t). 
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§  =  pitch- rate 

Then  sensor  #1  may  yield  the  measurement  (scalar) 
roll  angle  +  noise 

while  sensor  #2  may  yield  the  measurement  (vector) 
roll  angle  +  noise 
yaw  rate  +  noise 

and  sensor  #3  may  yield  the  measurement  (vector) 
roll  angle  +  noise 
pitch  angle  +  noise 
roll  rate  +  noise 
yaw  rate  +  noise. 

2,4  Measurement  Constraints 

As  we  have  indicated  in  the  introduction,  we  shall  assume  that  at 
each  instant  of  time,  t,  we  are  constrained  in  looking  at  only  one  of  the  data 
signals  available  from  the  sensors.  We  are  free,  of  course,  to  switch  from 
one  sensor  to  another. 

To  motivate  our  definitions  one  can  imagine  that  we  have  a  measurement 
selector  box  whose  output  z(t)  can  be  connected  directly  to  either  Zj^(t)  or 
^^(t),  ...  or  _Zj^(t).  This  is  illustrated  in  Figure  4. 

A  convenient  way  of  modelling  this  simple  switching  task  of  the 
measurement  selector  is  to  define  M  time  functions 

Vi(t),  v^Ct),  ,  .  .  ,  Vj^(t)  (2,16) 

with  the  following  properties 

a)  at  each  instant  of  time  Vj(t)(j  =  l,  2,  ,  ,  .  ,  M)  can  attain  either 
the  value  0  or  the  value  1 
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|l8-3-13S35 


Fig.  5.  Alternate  modelling  of  the  signal  z(t). 
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b)  If  V.  (t)  =  1,  then  v,  (t)  =  0,  for  k  =1,  2 . j-  1,  j+1,  .  .  .  ,  M 

J  ^ 

Mathematically,  we  can  define  the  switching  vector  v(t)  whose  components 
are 


v(t)  = 


v^(t) 

V2{t) 


(2.17) 


and 

iVj  (t)e[  0,  l] 

M 

I  V.(t)  =  1 

j=l 


(2.18) 


(2. 19) 


The  selected  data  signal  z(t)  can  then  be  written  as 


z(t)  =  Vj^(t)^j^(t)  +  +.  .  .  +  Vj^(t)^j^(t) 


(2.  20) 


with  (dim  meaning  dimension  of  column  vector) 

dimz(t)  =  dimz.(t)  when  v.(t)  =  1 


(2.21) 


As  shown  in  Figure  5,  we  can  then  model  the  measurement  selector  by 
multiplying  each  measurement  2j(^)  ''adding”  the  results.  Figure  6 

illustrates  how  z(t)  is  formed  from  individual  possible  measurements. 

2.  5  Cost  of  Observations 

As  we  have  indicated  in  the  introduction  one  can  associate  an  observa- 
tion  cost  to  each  one  of  the  M  possible  observations.  Such  a  cost  can  be 
used  to  reflect  that 
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Z(t) 


Fig.  6.  Illustration  of  the  relation  of  the  signal  z(t)  to  three  signals  Zj^(t), 
Z2(t),  Z2(t)  as  a  function  of  the  observation  variables  Vj(t),  v^it),  v^(t). 
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a)  Special  resources  or  instruments  may  be  required  to 
carry  out  a  specific  observation. 

b)  Special  computational  algorithms  (different  in  programming 
overhead  and/or  real-time  requirements)  may  be  required 
to  process  each  special  observation. 

For  this  reason,  we  shall  assume  that  there  is  an  inherent  cost 
that  must  be  taken  into  account  in  order  to  arrive  at  an  optimal  observa¬ 
tion  policy. 

We  shall  denote  by  ^j(^)  ^^e  per- unit- of  time  cost  of  making  the 
observation  z.(t)  at  time  t.  We  assume  that 


0  ^  2,  .  .  .  ,  M 


(2.22; 


Since  one  is  limited  to  a  specific  observation  at  each  instant  of  time,  then 
one  can  associate  with  each  observation  policy  v(t)  a  total  cost,  denoted  by 
q(v),  by 

T 


dt 


(2.  Z3 


M 

q(v)  =  r  ^  q^(t)v.(t) 

q(v)  then  represents  the  total  observation  cost  associated  with  the  use  of  the 
observation  strategy  v(t)  in  the  time  interval  t^  ^  t  ^  T. 

2,  6  Prediction  Requirements 

The  definition  of  an  optimal  observation  policy  during  a  time  interval 
t  ^  t  ^  T  cannot  be  made  on  the  basis  of  the  observation  cost  alone.  Usually 
one  makes  observations  upon  a  physical  plant  or  process  in  order  to  predict 
its  future  response.  (This  in  turn  may  be  used  for  control  purposes,  if  it 
turns  out  that  the  future  response  is  in  some  sense  unsatisfactory). 
Intuitively,  one  would  expect  that  the  accuracy  of  any  prediction  or  fbre- 
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casting  policy  will  depend  on  the  information  content  and  accuracy  of  the 

observations  that  have  been  already  made.  Hence,  an  optimal  observation 

policy  must  depend  (in  addition  to  the  cost  of  observation)  upon  the  accuracy 

of  the  prediction  for  which  observations  are  made. 

In  this  note,  we  shall  assume  that  the  purpose  of  observations  during 

a  time-interval  ^  ^  ^  T  is  to  predict  the  value  of  a  vector  w(t),  associated 

with  the  plant  variables,  at  some  value  of  time  t  =  T  ,  T  ^T,  where  T  -  T 

P  P  P 

is  the  length  of  the  prediction  interval. 

For  example,  consider  the  radar  measurements  carried  out  on  a 

ballistic  target.  Suppose  that  diverse  measurements  are  carried  out  during 

the  time- interval  [t^,  T].  Then  one  may  be  interested  in  predicting  at  some 

time  T  ,  T  >  T, 

P  P 

a)  the  position  vector  of  the  target 

b)  its  mass 

to  aid  in  decisions  involving  its  potential  threat  to  a  defense  site  and/or 
potential  interception. 

To  be  specific,  we  assume  that  w(t)  is  an  k-dimensional  vector, 
whose  components  summarize  important  plant  characteristics.  We  shall 
call  w(t)  the  important  plant  vector.  We  shall  assume  that  w(t)  is  linearly 
related  to  the  plant  state  vector  x{t)  by  the  equation. 

w(t)  =  D(t)x(t)  (2.  24) 


where  D(t)  is  a  known,  possibly  time- varying,  kxn  matrix. 

We  can  now  state  in  a  precise  manner  the  prediction  requirements 
of  our  problem.  Let  '^(t)  denote  an  estimate  of  w(t).  Then,  the  accuracy  of 
our  prediction  scheme  hinges  on 
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a)  whether  or  not  Efw(T  )  -  w(T  )]  =  0  i.  e.  whether 

—  p  “  p  — 

or  not  at  the  prediction  time  the  estimation  error  vector 

w(T  )  -  w(T  )  has  zero  mean. 

-  P  ~  P 

b)  the  value  of  the  mean  square  error 


J(T  )  =  E{(w(T  )-  w(T  ))'(w(T  )  -  w(T  ))}  (2.25) 

sr  r  IT  IT  ir 

A 

(the  smaller  J(T^),  the  more  accurate  the  prediction). 

2,  7  Statement  of  Optimization  Problem 

We  are  now  ready  to  formulate  in  precise  terms  the  optimization 
problem  under  consideration,  whose  solution  will  specify  the  optimal 
observation  program  defined  by  the  observation  vector  v(t).  Given  the  plant 


x(t)  =  A(t)x(t)  +  B(t)u(t)  +^(t);  (2.26) 

and  the  important  plant  vector 


w(t)  =  D  (t)x(t) 


(2.27) 


Given  the  sensor  signals 


z^(t)  =  C^.(t)x(t)  +  ^^(t);  j  =  l,  2,  .  .  .  ,  M  (2.  28) 

Let  t  ^  t  ^  T  be  a  fixed  time  interval  and  let  T  ,  T  ^  T  be  a  fixed  pre- 
o  P  P 

diction  time.  Determine  the  scalar  variables 


Vj(t),  V2(t),  .  .  .  ,  v^(t),  T] 

subject  to  the  constraints 

M 

Vj(t)e  [  0,  1} ;  y  Vj(t)  =  1 

j  =  l 


(2.29) 


(2.30) 
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and  a  prediction  algorithm,  such  that  if  denotes  an  estimate  of  w(T  ), 

given  an  observation  program  and  observations  during  t^  ^  t  ^  T,  then 


E{w(T  )-w(T  )}  =  0 

-  p  -  p  - 


(2.31) 


and  the  scalar  cost  functional 


J  =  a  J 


M 


j=i 


qj(t)Vj(t) 


dt  +  E{(^p)  -  w(Tp))'(w(Tp)  -  w(Tp))] 


=  aq(v)  +  J(T 


(2.  32) 


with  a  2:  0  (2.  33) 

is  minimized. 

We  remark  that  a  is  a  weighting  constant  that  reflects  the  relative 
importance  of  the  total  observation  cost 

M 


.M  =  J 


I 


Y  qj(t)Vj(t) 

j=i 


dt 


(2.34) 


with  respect  to  the  *’mean  square  error" 


J(T  )  =  E{(w(T  )-  w(T  ))'(w(T  )  -  w(T  ))} 

P  jJ  jJ  |J 


(2.  35) 


in  the  overall  cost  functional  J. 


18 


3. 


PREDICTION  ACCURACY  FOR  ANY  GIVEN  OBSERVATION  POLICY 


Let  v(t),tc:[t^,  T]  denote  any  fixed  observation  policy  during  the  time- 

interval  t  ^t^T  .  For  any  such  obse  rvation  pol  icy  one  can  determine  a 
o  p 

Kalman-Bucy  filter,  [5], which  is  ’'matched^’  to  the  observation  policy. 

3.1  State  Estimation 

Denote  by  the  estimate  of  the  plant  state  x(t),  given  an  observa¬ 
tion  program  v(t),  ^  T  ^  t  and  the  subsequent  observation  z(t),  t^  ^  t  ^  t, 

as  generated  by  the  Kalman-Bucy  filter.  The  subscript  v  is  used  to  stress 
the  dependence  of  the  estimate  upon  the  observation  policy  used. 

Let  "v^(t)  denote  the  state  estimate  error  covariance  matrix,  i.  e.  the 
covariance  matrix  of  the  state  estimation  error  x(t)  -  x^(t), 


E^(t)  =  E{  (x(t)  -  x^(t))(x(t)  -  x^t))'}  (3.1) 

It  can  be  shown  that  the  estimate  x^(t)  is  generated  by  the  solution  of  the 
stochastic  differential  equation  (Kalman- type  filter) 


^  x^(t)  =  A(t)x^(t)  +  S^(t) 


M 


1, 


y  V  (t)  c'.(t)  0.'^(t) 

j  —j  ~j 

j=i 


Vj(t)Cj(t)j  x^(t) 


(3.2) 


+  B(t)u(t);  X  (t  )  =  X 
— '  — '  —V  o  — o 


where  z(t)  is  the  actual  observation  signal  obtained  from  the  policy  v(t) 


M 

z(t)  =V  v.(t)z.(t)  (3.3) 

j  =  l 

The  error  covariance  matrix  is  the  solution  of  the  matrix  Riccati 

differential  equation 

'J'  because  the  essential  linearity  of  the  equations  is  not  affected  from  the  use 
of  a  specific  measurement  policy. 
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-  ^v(') 


M 

y  v.(t)C.'(t)^."V)C.(t) 

r=i 


(3.4) 


5:  (t);  r  (t  )  =  E 

— V  — V  o  — o 


We  remark  that,  for  any  given  v(t),  the  state  estimation  error  has  zero  mean; 


E{x(t)  -  x^(t)]  =  0  for  all  te[t^,  T] 


(3.5) 


3.  2  State  Prediction 

The  predicted  estimate  x(T  )  of  the  state  x(T  )  can  be  computed  from 

-  P  -  P 

the  state  estimate  x  (T)  by 


j  _  ^  B(r)u(r)dr  (3.  6) 


p  -V 


where  $(t,r)  is  the  transition  matrix  defined  by  A(t),  i.e. 


-^Mt.r)  =  A(t)$(t,r):  $(r,r)  =  I 


(3.  7) 


3,  3  Important  Parameter  Prediction 

The  predicted  estimate  w  (T  )  of  w(T  )  is  generated  by 
^  — v'  p  ~  p 

w  (T  )  =  D(T  )x(T  )  (3.  8) 

-V  p  -  P  -  P 

It  can  be  shown  that  this  leads  to  a  zero  mean  prediction  error,  i.  e. 


E{w(T  )  -  w  (T  )]  =  0 

—  p  — V  p  — 


(3.  9) 


Let  S  (T  )  denote  the  covariance  matrix  of  the  prediction  error  w(T  )  - 

— V  p  ^  —  P 

w  (T  ),  i.e. 

-V  p 


S  (T  )  =  E{  (w(T  )  -  w  (T  ))(w(T  )  -  w  (T  ))M  (3.10) 

— V  P  —  p  p  —  p  —V  p  ' 


Then  the  error  covariance  matrices  S  (T  )  and  I^(T  )  are  related  by 

—V  p  —  p  ^ 
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S  (T  )  =  D(T  )r  (T  )D'(T  ) 

— V  p  —  p  — V  p  —  p 


(3.11) 


But  the  predicted  state  error  covariance  matrix  T.  (T  )  is  related  to  the  state 

^  —V  p 

error  covariance  matrix  T  (T),  at  the  end  time  T  of  the  observation  interval,  by 


5:  (T  )  =  $(T  ,  T)i:  (T)  f  '(T  ,  T)  (3.  12) 

—V  p  ”  p  —V  —  p 

where  ^{t,T)  is  the  transition  matrix  defined  by  Equation  (3.  7). 

Hence,  from  Equations  (3.  11)  and  (3.  12)  we  conclude  that 
S  (T  )  =  D(T  )$(T  ,T)!:  (T)#'(T  ,  T)D'(T  ) 

-v'  p^  p'-'  p^  '~v'  '  p’  ^  p'  (3.13) 

Let  us  now  recall  that  our  prediction  accuracy  was  measured  by 
the  ’’mean  square  error”  (See  Equation  (2.35)) 


J  (T^)  =  E[(w(T^)  -  w^(Tp))'(w(T^)  -  v^^(T^))}  (3.14) 


V  p 


By  a  matrix  identity,  J(Tp)  can  also  be  written  as 

J(T  )  =  E[  tr  (w(T  )  -  w  (T  ))(w(T  )  -  w  (T  ))'  } 

'  p  I  ~  P  p  —  p  —V  p  J 

'] 


=  tr 


=  tr 


Ef  (w(T  )  -  w  (T  ))  (w(T  )  -  w  (T  )) 

—  p  —V  p"  '  p  —V  p 


S  (T  ) 
-v'  p^ 


•] 


and  so,  in  view  of  Equation  (3,  13),  J(Tp)  is  given  by 


J(T  )  =  tr 
P 


D(T  )  $(T  ,  T)i:  (T)  $'(T  ,  T)D'(T  ) 

— '  p  — '  p  —V  —  p  —  p 


(3.  15) 


(3. 16) 


We  can  now  see  that  any  given  observation  policy  v(t),  te[t^,  T],  defines  a 
state  error  covariance  matrix  r^(T),  by  the  solution  of  the  matrix  Riccati 
differential  equation(3.  4)  and,  hence,  a  value  of  J(Tp)  from  Equation  (3.16). 
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4.  REFORMULATION  OF  THE  OPTIMIZATION  PROBLEM 

The  above  discussion  points  out  that  the  optimization  problem  stated 

in  Section  2.  7  can  be  reformulated  as  follows: 

Given  the  matrix  Riccati  differential  equation 

M 

-^E(t)  =  A(t)Z(t)  +  E(t)A'(t)  +H(t)  -  Z(t)(  y  v.(t)C'(t)0y(t)C.(t))r(t); 


jij  J  -J  -J  -J  /- 


r(t  )  =  r 

— '  O  — O 


(4.1) 


(The  elements  of  r!(t)  are  viewed  as  the  state  variables  and  the  Vj(^) 
control  variables) 

Given  the  constraints  on  the  •  •  •  >  . 


Vj(t)e{0,  1],  for  all  te[t^,  t]  (4.2) 

M 

y  v.(t)  =  1,  for  all  te[t^,  T]  (4.3) 

j=l 

Find  the  optimal  Vj'’'(t)  such  that  the  cost  functional,  with  t^,  T  fixed 


J  =aj 


M 

y  q  (t)v  (t) 

1=1  J 


dt  +  tr 


D(T  )$(T  ,T)i:(T)$'(T  ,T)D'(T  ) 

-p-p--p  -p 


(4.4) 


is  minimized. 

We  remark  that  this  is  a  deterministic  optimal  control  problem.  Since 
the  dynamic  constraints  (4.  1)  are  naturally  expressed  via  a  matrix  differential 
equation,  one  can  obtain  the  solution  through  the  use  of  the  matrix  minimum 
principle  (Athans,  [6  ]). 
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5. 


APPLICATION  OF  THE  MATRIX  MINIMUM  PRINCIPLE 


Let  P(t)  denote  an  nxn  costate  matrix  associated  with  the  covariance 
matrix  T  (t). 

Define  the  scalar  Hamiltonian  function  for  the  posed  optimization 
problem  as  follows: 


or 


H  =  H(E(t),  P(t),  v.(t),t). 


=  a 


M 

y  q.(t)v  (t)  +  tr[E(t)P'(t)] 
J  J  ~  “ 


M 

H  =  a  y  qj(t)v.(t)  +  tr  [A(t)E(t)P'(t)] 

j  =  l 


(5.1) 


+  tr  [E(t)A'(t)P'(t)]  +  tr  [H  (t)P'(t)] 


(5.2) 


+  tr 


'M 


E(t)n  v.(t)C.'(t)0."\t)C.(t)  I  E(t)P'(t) 


5>  1  Conditions  for  Optimality 

Let  Vj'5'(t)  characterize  the  optimal  observation  policy,  E'!'(t)  the 
resultant  state  error  covariance  matrix,  and  P^i"(t)  the  corresponding  costate 
matrix.  Then  the  following  properties  are  true. 

Canonical  equations: 


ap(t) 


=  A(t)E*(t)  +  i:^i<(t)A'(t)  +  S(t) 


=  S=:<(t) 


M 

y  V  =:=(t)  C/(t)0.‘\t)  C.(t) 
--  J  “J  “J  “J 

j  =  l 


(5.3: 


E=:<(t) 
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■Jt  -  ■  ^E(t) 


,,  =  -  R':=(t)A{t)  -  A'(t)P*(t) 


M 


+  R<(t)r=Mt)(  V  V  >.Mt)c'.(t)Q  ■\t)c.(t) 

-  -  \  - '  J  -j  -j  -j 

/ 

(M 

Y  Vj*(t)c. '(tR"\t)c.(t)  I  '  E=i<'(t)R;=(t) 

u=l 


(5.4) 


Boundary  Conditions 
At  t  =  t 


7:>Ht  )  =  E 
—  o  — o 


(5.  5) 


At  t  =  T 


R:=(T)  =  t°(Tp)J'(Tp,  T)  E(T)£'(Tp,  T)D'(Tp)] 


=  $ '(Tp,  T)D'(Tp)D(Tp)$(Tp,  T) 
Hamiltonian  Minimization 


(5.  6) 


The  inequality 


H(p(t),  R^(t),  v.^:^(t),  t)  ^  H(E^:^(t),  ^:^(t),  v.(t),t)  (5.7) 


or  (see  equation  (5.2)). 


M 


Cl  )  q.{t)v.»:<{t)  -  tr 

j 


1=1  J  J 


/  M 


E=:<(t)  Y  V  «(t)c/(t)0  "\t)C.(t)  E»:=(t)Ri<tt) 

-  \  J  -J  -J  -J  /  -  - 


M 

^  a  Y  q.{t)v.  (t)  -  tr 

j 


M  J  J 


/  M 

(5 


E*(t)(  Y  V  (t)C'.(t)0."\t)Cjt)  1  E=f<(t)R^1:t) 
i'='l  J  -J  -J  -J 


(5.8) 
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M 


must  hold  at  each  te[t  ,  T]  and  for  all  v.(t)e{0,  l],  7  v.{t)  =  1 


t-J  1 

j  =  l 


o  J 

5.2  Implications  of  Necessary  Conditions 
The  properties  of  the  trace  function  can  be  used  to  simplify  the 
inequality  (5.  7).  Since 


tr 


M 


z:*(t)(  Y  v.{t)c/(tB,'\t)c,(t)  1  i:>:<(t)R^'{t) 


j  -j  -j  -j 


M 

Y  V. 

-  J  '  -J  ■  -J  '  -J 

L\j=i 


(t)c / (t)0  ■  \t)c, (t)  1  i:=:<(t) p* ' (t)E>:=(t) 


M 


=  Y  V  (t)  tr  [c'(t)0."\t)C.(t)5:^:<{t)P>l<'(t)i:*(t)] 
^  J  “J  J  “J  —  —  — 

j=l 


(5.  9) 


define,  for  notational  simplicity,  the  (symmetric  at  least  positive  semi- 


definite)  matrices  Lj(t)  by 

L.(t)  =  C. '(t)0."\t)C.(t) 
-J  -J  -J  -J 


(5.10) 


Using  the  above,  the  inequality  (5.  7)  can  be  written  as 
M 

y  vy(t)|c 


aqj(t)  -  tr  [Lj(t)p:<(t)^'(t)E*(t)] 


j=l 


M 

^  Y  v.(t) 

J 

j=i 


aq.  (t )  -  tr  [  L.  (t  )E=:=  (t )  r:=  '  (t )p  (t )  ] 


(5.11) 


One  can  think  of  the  L.(t)  as  the  matrices  that  are  related  to  the  *'signal- 
to-noise’*  ratio  of  the  ^  j-th  possible  observation  at  time  t. 
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Define  the  switching  functions  Sj^‘'(t),  j=l,  2,  .  .  .  ,  M 


s.*(t)  =  aqj(t)  -  tr[L.(t)C>:<(t)R:='(t)S>!<(t)] 

In  view  of  the  constraints  on  the  v.(t)  we  can  conclude  that 

J 


Vj*(t)  =  1  if  ^  ^  =  1.  2,  .  .  .  ,  M,  kT^j 


v.(t)  =  0  otherwise 


(5.12) 


(5.  13) 
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6. 


NUMERICAL  SOLUTION  OF  THE  TWO  POINT  BOUNDARY 
VALUE  PROBLEM 


The  equations  that  define  the  properties  of  the  optimal  observation 
policy,  derived  by  the  matrix  minimum  principle,  and  stated  in  Section  5 
represent  a  nonlinear  two  point  boundary  value  problem.  Since  we  deal 
with  nonlinear  matrix  differential  equations,  techniques  such  as  Newton's 
method  are  difficult  to  apply  since  they  involve  the  computation  and  inversion 
of  a  fourth  order  tensor  quantity.  On  the  other  hand,  standard  gradient 
techniques  cannot  be  used  due  to  the  "on-off"  nature  of  the  observation 
variables 

A  technique  which  can  be  used,  ina  relatively  straight  forward  manner, 
is  the  H-minimal  technique  suggested  by  Kelley  [11].  For  the  sake  of  completeness 
the  algorithm  is  summarized  in  the  flow  chart  of  Figures  7,  1  to  7.  4.  In  the 
construction  of  this  algorithm  the  following  properties  (which  are  easy  to  verify) 
have  been  used 

.  The  costate  matrix  P^l-(t)  is  symmetric 

and  at  least  positive  semidefinite 

.  The  covariance  matrix  E'l=(t)  is  symmetric 

and  at  least  positive  semidefinite 

There  is  no  guarantee  of  convergence  of  the  H-minimial  algorithm 
in  general.  Also,  the  nonlinear  nature  of  the  matrix  differential  equations 
involved  precludes  any  a  priori  knowledge  of  existence  of  locally  optimal 
observation  strategies  in  addition  to  the  globally  optimal  one.  Additional 
research  is  currently  underway  to  determine  convergence  properties  and  the 
use  of  alternate  computational  algorithms  for  the  solution  of  the  2-point  boundary 
value  problem. 


27 


INPUT 


n  -  dim  A,  t  ,  T,  T 
o  p 

A(t).  5^.  M,  C, .  .  .  ,  Cj^  (C^  ^  X  n) 


m,  ,  m^ . m.  -  ,©  ,(t) . (0.  -  m  Xm.) 

I  Z  M  —  1  — M  ~J  J  J 


D(t),  «,  qjU),  .  .  .  •  €,  i_  J 


Fig.  7.  Structure  of  the  digital  computer  algorithm  for  the  determination 
of  the  optimal  observation  policy. 
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18-3-13538 


k  =  0 


Fig.  7.  Continued. 
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30 


Fig.  7,  Continued. 
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Fig.  8.  Generation  of  optimal  measurement  z=:  (t). 
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7. 


IMPLEMENTATION 


It  is  important  to  recognize  that  the  determination  of  the  optimal 
observation  policy  v*(t),  te[t^,  T],  is  an  off-  line  problem.  That  is,  the 
solution  of  the  two- point-boundary  value  problem  does  not  have  to  be  done 
while  measurements  are  being  made.  The  reason  is  that  the  actual  measure¬ 
ments  ^j(t)  not  enter  in  the  equations  of  the  optimization  problem  whose 
solution  determines  the  optimal  observation  policy;  rather,  it  is  only  the 
statistics  of  the  problem  and  the  plant  dynamics  that  are  relevant, rather 
than  the  measurements  themselves. 

Once  the  optimal  observation  policies  v^'l'(t),  V2*(t),  .  .  .  , 
been  computed  (off-line!),  then  one  can  implement  the  ’’matched’’  Kalman 
Bucy  filter  and  predictor  which  operates  upon  the  actual  measurements  to 
generate  the  optimal  estimate  w5l^(t)  of  the  important  plant  parameter  vector 
w(t)  at  any  instant  of  time  and  at  the  prespecified  prediction  time  T^. 

Figure  8  shows  the  generation  of  the  actual  signal  z>!^(t)  that  drives 
the  Kalman  filter  once  V"i'(t)  has  been  obtained  (compare  with  Figure  5)  from 
the  naturally  available  measurements  Zj^(t),  .  .  .  >_£2^(t). 

M 

=  y  v.>^(t)z.(t)  (7.1) 

j=i 

The  optimal  state  error  covariance  matrix  ^*(t) 

can  be  computed  off-line,  once  v*(t)  has  been  obtained  by  solving  the  matrix 
Riccati  equation  (compare  with  Equation  (3.4)), 
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Fig.  9.  Structure  of  the  Kalman-type  filter  which  is  ’’matched" 
to  the  optimal  observation  policy. 


18-3*13543 


x*(t) 

o— 


(from  Kolmon  Bucy 
filter) 


♦(Tp,t) 

~r~ 

I 


I 

j 


PRECOMPUTED 


xMTpIt) 


w«  (Tpit) 


INSTANTANEOUS 


6 

j  ♦(Tp.rlBlTlulrldr 

( precomputed ) 


PREDICTED  ESTIMATE 
OF  w(Tp)  GIVEN 
MEASUREMENTS 
UP  TO  TIME  t  <  T 


Fig.  10.  Structure  of  the  predictor. 
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(7.3) 


-^E^(t)  =  A(t)p(t)  +p(t)A'(t)  +H(t) 

M 

-  E*(t)  V  v.>^(t)C.'(t)^.‘\t)C.(t)  E>^-(t^)=r^ 

j=l 

The  structure  of  the  Kalman  filter  that  generates  the  state  estimate  x*(t) 

=  x^^^(t)  (7.4) 

is  shown  in  Figure  9.  The  Kalman  gain  matrix  &;=(t) 

M 

G=Mt)=p(t)  ^  vyMt)C.(t)q.-\t)  (7.5) 

j=l 

can  be  computed  off-line  once  v^'(t)  and  have  been  found. 

The  diagram  of  Figure  9  helps  to  visualize  how  the  optimal  observa¬ 
tion  variables  Vj^^=(t),  .  .  .  ,  determine  the  signal  to  be  subtracted  from 

z'^'(t),  generated  by  the  selector  of  Figure  8.  One  can  obtain  the  instantaneous 
estimate  w^{t)  by  simply  multiplying  the  state  x'!'(t)  of  the  Kalman  Bucy  filter 
by  the  known  matrix  D(t). 

As  the  actual  observations  are  being  made,  one  can  compute  the 

predicted  estimate  w^'(T  )  t)  of  w(T  )  given  observations  only  up  to  time  t 

P  P 

(t  ^  T).  Figure  10  illustrates  the  on-line  computations  required  to  generate 
this  predicted  estimate. 
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8. 


CONCLUDING  REMARKS 


A  digital  computer  program  is  currently  being  developed  to  solve 
the  two  point  boundary  value  problem  discussed  in  Section  6.  Its  performance 
as  well  as  numerical  examples  will  be  reported  in  the  future. 

Extensions  of  the  basic  ideas  to  the  nonlinear  dynamics  case  are 
currently  under  investigation.  The  approach  consists  of  matching  an  ex¬ 
tended  Kalman  filter  to  a  particular  observation  program  and  then  attempting 
to  optimize  the  observation  program.  However,  in  the  nonlinear  case  the 
situation  is  much  more  complex,  since  in  the  extended  Kalman  filter  the 
(pseudo)  error  covariance  matrix  cannot  be  accurately  precomputed  and, 
in  fact,  it  is  coupled  to  the  estimation  equation.  For  this  reason,  the 
optimal  observation  program  has  to  be  computed  and  updated  on  line.  This 
may  present  excessive  on-line  computational  requirements.  The  projected 
research  effort  will  be  focused  on  techniques  that  have  less  severe  on-line 
computational  requirements;  however,  these  may  yield  suboptimal  measurement 
strategies.  Hence,  trade-off  studies  will  be  necessary  in  order  to  establish 
concrete  results  in  this  important  class  of  problems. 
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